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ABSTRACT 

A study was conducted to describe Ohio secondary 
vocational education teachers' use of student assessment information 
in making instructional decisions. The target population for the 
study was all teachers who taught in secondary vocational education 
programs in Ohio public schools. A questionnaire was mailed to a 
random sample of 393 secondary vocational education teachers 
stratified by program area; 290 usable questionnaires (74 percent 
response rate) were received. The questionnaire asked teachers to 
indicate the extent to which they use information derived from 6 
types of assessment methods in addressing 10 different instructional 
decisions. The six types of assessment methods were as follows: 
objective paper and pencil items; informal observations; standardized 
test scores; performance assessments; portfolios; and essay type 
items. The 10 decision areas in which assessment results are commonly 
used were the following: plan for instruction; diagnose student 
weakness; monitor student progress; communicate achievement; motivate 
students; evaluate instruction; evaluate instructional materials; 
group students; encourage self-assessment; and assign grades. 

Teachers rated information provided from performance assessments as 
being of more use in addressing day-to-day classroom decisions than 
information obtained from the other five assessment methods. Teachers 
also relied heavily on objective paper and pencil methods and 
informal observations. Recommendations included the following: (1) 
because teachers often use performance assessments, teacher 
preservice education curriculum should pay particular attention to 
the development of competence in the use of performance assessments; 
(2) the Ohio Department of Education should encourage teachers to use 
standardized test results; (3) further research should be conducted 
in the area of the use of portfolios in assessment; and (4) further 
research should be conducted on vocational education teachers' 
attitudes toward standardized testing and the use of authentic 
assessment methods. (Contains 18 references.) (KC) 
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There has been widespread concern voiced throughout the country about low 
achievement among the nation's school children. The National Commission on 
Excellence in Education (1983) declared the United States to be a nation at risk, awash in 
a rising tide of mediocrity. Since that time there has been no lack of reports criticizing 
curriculum, administrators, teachers, parents, and students. Concerns for international 
competitiveness, renewed calls for restructuring, and the accountability movement has 
prompted a search for the means to achieve excellence in our schools. 

In response to requests for change, educational organizations around the country 
have sought ways to document the effectiveness of their educational programs. .State 
after state has sought to initiate mechanisms which serve to promote accountability for 
educational outcomes. California, Connecticut, Iowa, Kansas, Maryland, and New Jersey 
have taken measures to make their educational systems more accountable for student 
outcomes (McCrt iin, 1990). The Action Plan for Accelerating the Moderniz.ation of 
Vocational Education in Ohio (Ohio Department of Education, 1990) required that a 
comprehensive accountability and evaluation system be developed and integrated with 
the statewide management information system. 

On the federal level, the Carl D. Perkins Vocational and Applied Technology 
Education Act of 1990 required accountability of all states who accept federal funds to 
support vocational programs. This was to be achieved through a system of specified 
performance measures and standards which track both academic and occupational 
competency gains. 

Testing at all grade levels has increased due to accountability demands, the 
information demands of objective-based instructional systems, and competency-based 
evaluation trends (Green & Stager, 1986). Norm-referenced standardized tests have 
become the tool of choice in obtaining data for decision making at the state and local 
level. Those in favor of standardized testing have asserted that such tests promote high 
standards for learning, facilitate more accurate placement decisions, yield information for 
the improvement of curriculum and instruction, and help the public hold schools 
accountable (Dorr-Bremme, 1983). 

Standardized testing in American schools has been and continues to be a subject 
of controversy from the local to the national level. Such tests have proven to be a time 
and cost effective means for measuring achievement however questions have arisen when 
significant emphasis was placed on the outcomes of the.se assessment methods. Worthen 
and Spandel (1991) have implied that standardized tests do have value when used 
correctly but provide only part of the picture and have their limits. 

Statewide assessment cannot attempt to measure and thereby reflect all that local 
schools are able to achieve in terms of student outcomes. It has become necessary for 
schools to measure the attainment of their unique educational objectives (Perlman, 1991 ). 
Public outcries for school reform have increased the pressure on teachers to not only 
construct tests to assess students mastery of skills but to also promote more rigid 
standards for student accountability (Carter, 1984). 

It has become necessary to document that students have more than seat time to 
account for the learning that was to have taken place in the classroom or laboratory. How 
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teachers use assessment information in the classroom and whether its use is effective can 
play a major role in enhancing and documenting both instruction and learning. How are 
vocational education teachers using student assessment data? Literature has revealed 
very little about the assessment practices of this group of teachers. Only recently has the 
measurement community focused research emphasis upon teacher assessment practices in 
the academic classroom. Research conducted on the quality and effectiveness of non- 
vocational teacher training in measurement and assessment has implied that these 
teachers may not be gaining the appropriate skills necessary for effective use of 
assessment. If this is true for academic teachers what of the assessment skills of 
vocational education teachers? 

The drive towards increased accountability has placed a demand on teachers to 
improve instruction and promote higher levels of achievement. To achieve these goals it 
is important to generate and utilize information which accurately measures effectiveness 
of instruction and the outcomes of learning. The more accurately teachers judge student 
achievement and performance the more effective they will be in directing student 
learning. An increased understanding of teacher practices in assessment use should aid in 
making more intelligent decisions in directing pupil progress toward worthwhile 
educational outcomes. 



Purpose / Objectives 

The purpose of this descriptive-correlational study was to describe Ohio 
seconda^ vocational education teachers' use of student assessment information in 
making instructional decisions. The specific objectives of this study were to: 

1 . Describe vocational education teachers' perceptions of their use of student 
assessment data for making instructional decisions. 

2. Describe vocational education teachers' perceptions of their competence in the 
assessment process. 

3. Describe vocational education teachers' perceptions of their attitude towards the 
assessment process. 

4. Examine the relationship between teacher use of assessment information and 
level of competence in the assessment process, and their attitudes towards 
assessment. 

5. Determine the proportion of variance in vocational education teachers' perceived 
use of assessment information in instructional decision making that could be 
explained by the independent variables of attitude towards assessment and 
competence in the assessment process. 

Methods / Procedures 

The target population for this study was all teachers who taught full time, 
secondary vocational education programs in Ohio public schools. The study utilized a 
random sample of 393 secondary vocational education teachers stratified by program 
area. 

A mailed questionnaire was designed by the investigator for use in measuring the 
variables of interest. To obtain a measure of the dependent variable, teacher use of 
student assessment information, respondents were asked to indicate the extent to which 
they use information derived from six types of assessment methods in addressing 10 
different instructional decisions. The six types of assessment methods used in the study 
included; objective paper and pencil items, informal observations, standardized test 
scores, performance assessments, portfolios, and essay type items. The 10 decision areas 
where assessment results are commonly used were identified as: plan for instruction. 
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diagnose student weakness, monitor student progress, communicate achievement, 
motivate students, evaluate instruction, evaluate instructional materials, group students, 
encourage self-assessment, and assign grades. 

Teacher competence in the assessment process was measured with a series of 
competency statements based on "Standards for Teacher Competence in Educational 
Assessment of Students" (American Federation of Teachers, National Council on 
Measurement in Education, & National Education Association, 1990). A 5-point Likert 
scale, which ranged from "not competent" to "extremely competent", was used. Teacher 
attitudes towards assessment was measured using a semantic differential scale. The scale 
was comprised of nine bi-polar adjectives which described the concept "assessment". 

A panel of experts was used to establish content and face validity. The 13 
member panel was comprised of university faculty, doctoral students, vocational 
education teachers, and state department of education staff. Following a review of the 
instrument, recommendations provided by the panel were incorporated into the 
instrument where appropriate. 

A revised instrument was pilot tested for reliability using a subsample of the 
population not selected for participation in the main study. A test-retest procedure was 
used with a two week interval between implementation. The results were compared for 
percent agreement with values that ranged from .64 to 1 .0 for each item. Items were said 
to be in agreement if the score from the initial test was no more than ± 1 .0 of the score on 
the retest. Measures of internal consistency were calculated from data obtained from the 
first phase of the test/retest procedure. Cronbach's Alpha ranged from .8 1 - .96 with a 
level of significance established a priori at .05. 

Usable questionnaires were received from 290 participants during the six week 
data collection phase. With 100 participants not responding, a 74% response rate was 
achieved. Ten percent of the non-respondents were randomly selected and interviewed 
by phone. Differences between non-respondents and respondents on the dependent 
variable were examined with a t-test. No significant differences were found between 
groups on the dependent variable or on the selected teacher characteristics. 

Pearson's r coefficient, was used to summarize the magnitude and direction of the 
relationship between variables. The conventions by Davis (1971) were used to describe 
the measures of association. Semi-partial, simultaneous multiple regression analysis was 
used to determine the variance in use of assessment information that was explained by 
selected independent variables. 



Results / Findings 

The descriptive statistics related to use of assessment information from each of 
the six assessment methods are presented in Table 1 . Teachers rated information 
provided from performance assessments (M=4.281 as being of more use in addressing 
day to day classroom decisions than information obtained from the other five assessment 
methods. This stands in slight contrast to Stiggins and Conklin (1992) who found that 
academic teachers placed more reliance on their own objective type assessment activities 
rather than on performance assessments for addressing instructional decisions. Given 
that vocational education teachers use a competency based curriculum, it was not 
surprising to the researcher that a performance based method of assessment was of more 
use than objective methods. 
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Table 1 

Descriptive Statistics for the Use of Assessment Information Generated From Six Assessment Methods (n=290)^ 
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Prior studies in assessment use found that teacher made objective tests and 
informal observations were the assessment methods upon which academic teachers 
primarily relied (Dorr-Bremme and Herman, 1986; Stiggins & Bridgeford, 1985; 
Gullickson, 1984). The results from this research study, as indicated in Table 1, 
corroborate the findings from previous studies and supported the conclusion that 
vocational education teachers placed a heavy reliance on the use of objective paper and 
pencil methods (M =4.10) and informal observations (M =3.76). 

Portfolios (M=2.74) and essay type assessments (M=2.53) were not found to be of 
much use for generating information in decision making (Table 1). Although portfolios 
continue to be promoted in education circles today vocational education teachers did not 
give particular emphasis to the information generated. Such a low reliance on portfolios 
may be accounted for by the relative newness of the assessment method. Vocational 
teachers may have yet to grasp the importance of a long term assessment method and 
build an assessment method of this type into their overall assessment system. 

Standardized test scores (M=2.26) were found to be the least used source of 
infomation for decision making (Table 1). This study supported the findings from other 
studies (Goslin, 1967; Yeh et al., 1981; Green, 1990; and others) regarding the lack of 
use of stpdardized test scores in addressing educational decisions. After years of 
standardized testing, the use of standardized test scores by vocational teachers continues 
to be minimal. 

The frequency distribution of vocational education teachers' competency scores is 
presented in Table 2. Vocational education teachers in this study reported that they 
perceived themselves to be moderately to very competent in the assessment process. 

Eight percent considered themselves to be extremely competent in the assessment 
process, 63% to be very competent, and 29% to be moderately competent. These results 
were congruent with findings by Gullickson and Hopkins (1987) which described 
teachers as being comfortable in their knowledge of assessment. Dorr-Bremme (1983) 
also concluded that teachers perceived their use of assessment techniques as accuratel\ 
measuring the effects of their instruction. 

Table 2 

Frequency Distribution of Summed Teacher Competency Scores (n=290^ 



Scale Value 


Summed Score 


f 


% 


Not Competent 


26-38 


0 


0.00 


Slightly Competent 


39 - 64 


2 


.69 


Moderately Competent 


65-90 


84 


28.97 


Very Competent 


91-116 


182 


62.76 


Extremely Competent 


117- 130 


22 


7.59 


|Mean= 97.24 


SD= 12.89 


Minimum= 


61 Maximum= 130 



An analysis of teacher a'.titude towards assessment is presented in Table 3. The majority 
of teachers (59%) reported having a positive attitude towards assessment (M=41.6). 
Thirty percent of the teachers reported being neutral in their attitude towards assessment 
and 12% percent perceived themselves to have a negative attitude towards assessment. 
The generally positive attitudes found by this study are in agreement with current 
findings (Green, 1990 «fe Green and Stager, 1986) where opinions of both experienced 
and preservice academic teachers towards classroom assessment were positive. 



Table 3 

Frequencies for Teacher Attitude Towards Assessment Scores (n=290^ 



Summed Attitude Scores 


f 

% 




9- 13 


Most negative attitude 


1 


.34 


14-22 




1 


.34 


23-31 




33 


11.38 


32-40 


Neutral 


86 


29.66 


41-49 




126 


43.45 


50-58 




43 


14.83 


59 - 63 


Most positive attitude 

Mean= 41.6 


0 


.00 



The correlational analysis between competence in assessment, attitude towards 
assessment, and use of individual assessment methods is presented in Table 4. 
Competence in assessment had a moderately positive association with teachers' use of 
information from objective paper and pencil methods and performance assessments and a 
low positive association with use of informal observations. Attitude towards assessment 
had a low association with the use of performance assessments. Use of portfolios, 
standardized tests, and essay items had only a negligible association with competence in 
and attitude towards assessment. 

Table 4 

Intercorrelations Between Selected Independent Variables and Assessment Methods 
(n=290) 





Intercorrelations 










X2 


Yl 


Y2 


Y3 


Y4 


Y5 


Y6 


Competence (X 1 ) 


.27 


.30 


.05 


.32 


.20 


.17 


.18 


Attitude (X2) 


1.00 


.17 


.05 


.26 


.10 


.11 


.12 


Objective (Yl) 




1.00 


.15 


.26 


.23 


-.04 


.27 


Standardized (Y2) 






1.00 


.03 


-.13 


.30 


.29 


Performance (Y3) 








1.00 


.51 


.21 


.10 


Informal (Y4) 










1.00 


.05 


.05 


Portfolio (Y5) 












1.00 


.28 


Essay (Y6) 














1.00 



Regression analy.ses are found in Tables 5-7. Competence in as.sessment was 
found to contribute 7.1% of the variation in teachers u.se of objective paper and pencil 
methods (Table 5), 5.7% in u.se of performance as.sessment (Table 6), and 3.9%' in use of 
informal observation (Table 7). Attitude towards assessment was found to explain only 
3% of the variation in teachers u.se of performance assessments. 



Table 5 

Semi-Part ial Regression of Use of Ob)ective Paper and Pencil Assessment Methods on 
Selected Characteristics (n = 290) 



Variables 


sR2 


b 


t 


D 


Competence 


.071 


.151 


4.78 


<.001 


Attitude 


.007 


.076 


1.49 


.138 


(Constant) 




18.77 






Standard error = 6,637 




Adjusted Sp' = 


.105 




r2 = .115 




For model: F = 


12.36, p<.(K)l 


Table 6 










Semi-Partial Regression of Use of Performance Assessment on Selected Characteristics 


(n= 290) 


Variables 


sR2 


b 


t 


E 


Competence 


.057 


.125 


4.46 


<.001 


Attitude 


.030 


.146 


3.23 


.001 


(Constant) 




24.14 







Standard error = 7.726 
r2 = ,201 



Adjusted = .169 
For model: F = 6.32, p<.0()l 



Table 7 

Reercssiv^n of Use of Informal Observation on Competence in Assessment (n= 290) 



Variables 


-r2 b 


t 


E 


Competence 


.039 .090 


3.40 


<.001 


(Constant) 


32.230 







Standard error = 5.812 Adjusted r^ = .035 

p- = .039 For model: £=11 .54, p<.0()l 



Conclusions / Recommendations / Implications 

Through a review of literature, the findings of this study, and the subsequent 
conclusions the researcher has proposed the following recommendations: 

1 . Given the competency based nature of the programs in vocational education, and 
given the findings from this study which document the high level of use of 
performance assessments by vocational education teachers, it is recommended 
that teacher preservice assessment curriculum pay particular attention to the 
development of competence in the use of performance assessments. 



2. Teachers are not using standardized test results yet it is expected that they use the 
results of current and upcoming standardized assessments for enhancing 
instruction and learning. Since the Ohio Department of Education is taking a 
leadership role in promoting the use of standardized assessment, it should also be 
responsible for promoting activities which will motivate and assist teachers in 
using standardized test information. 

3. The use of portfolio assessment is currently being promoted in education circles 
as a means to complement point-in-time assessment techniques. Yet, the use of 
portfolio assessment was not shown to be of much use to vocational education 
teachers nor was there a relationship with any of the independent variables 
investigated. The researcher believes that vocational education teachers may 
only be in an awareness stage in terms of their adoption of the portfolio 
assessment method. It is recommended that research be conducted to further 
investigate the use of portfolio assessment in vocational education settings. 

4. It was concluded that attitude towards the overall assessment process contributed 
little to understanding the use of assessment. It is recommended that future 
research focus on an examination of attitude towards use of specific assessment 
methods. Given the current trends in assessment it would be appropriate to focus 
research on vocational education teachers' attitude towards standardized testing 
and the use of authentic assessment methods. 

5. Overall competence in assessment explained only a small proportion of the 
variation in teachers use of three of the most used assessment methods. It is 
recommended that a competency measure oriented toward a specific assessment 
method be used to more clearly identify specific strengths and weaknesses 
related to the use of that particular assessment method. 



References 

American Federation of Teachers, National Council on Measurement in Education, & 
National Education Association. (1990). Standards for teacher competence in 
educational assessment of students . Washington, DC: Authors. 

Carter, K. (1984). Do teachers understand principles for writing tests? Journal of Teacher 
Education . 35(6), 57-60. 

Davis, J.A. (1971). Elementary survey analysis . Englewood, NJ: Prentice-Hall 

Dorr-Bremme, D.W. (1983, October). Assessing students: Teacher's routine practices and 
reasoning. Evaluation Comment . 6(4), 1-12. 

Dorr-Bremme, D.W., & Hemian, J.L. (1986). Assessing student achievement: A nrofile 
of classroom practices . Los Angeles: University of California, Center for the 
Study of Evaluation. (ERIC Document Reprouuction Service No. LD 338 691 ). 

Goslin, D.A. (1967). Teachers and te.sting . New York: Russell Sage Foundation. 

Green, K.E. (1990, April). Differences between teachers and students in opinions ahmji 
testing and test use . A paper presented at the annual meeting of the National 
Council on Measurement in Educatior., Boston, MA. (ERIC Document 
Reproduction Service No. ED 318 775). 



O 

ERIC 



lO 



Green, K.E., & Stager, S.F. (1986). Measuring attitudes of teachers toward testing. 

Measurement andEvaluation in Counseling and Development . 12(3), 141-150. 

Gullickson, A.R. (1984). Teacher perspectives of their instructional use of tests. The 
Journal of Educational Research . 77(4). 244-248. 

Gullickson, A.R., & Hopkins, K.D. (1987). The context of educational measurement 
instruction for preservice teachers: Professor perspectives. Educational 
Measurement: Issues and Practice . 6(3). 12-16. 

McCaslin, N.L. (1990). A framework for evaluating loc^vocational education prograrr s. 
Columbus, OH: The Ohio State University, Center on Education and Training for 
Employment. 

National Commission on Excellence in Education. (1983). A n ation at risk: The 
imperative for educational reform . 

Ohio Department of Education. (1990, June). Ohio's future at work: Action plan for 

accelerating the modernization of vocational education in Ohio . (Available from 
[Ohio Department of Education, Division of Vocational and Career Education, 
Columbus, OH]). 

Perlman, C.L., (1991, April). Developing a comprehensive assessment program . Paper 
presented at the annual meeting of the American Educational Research 
Association, Chicago, IL. (ERIC Reproduction Document Service No. ED 330 
715). 

Stiggins, R.J. & Bridgeford, N.J. (1985). The ecology of classroom assessment. Journal 
of Educational Measurement . 22(4), 271-286. 

Stiggms, R.J. & Conklin, N.F. (1992). In teacher's hands: Investigating the practices of 
classroom assessment . Albany, NY: State University of New York Press. 

Worthen, B.R. & Spandel, V.(1991). Putting the standardized test debate in perspective. 
Educational Leadership . 4§(5), 65-69. 

Yeh, J.P., Herman, J.L., & Rudner, L.M. (1981). Teachers and testing: A survey of test 
u^ (Report No.CSE-R-166), Los Angeles, CA: California University, Center for 
the Study of Evaluation. (ERIC Reproduction Document Service No. ED 218 
336). 




11 



